Skip to content

feat: Enable expressions as default value in lead/lag function#22134

Open
sandugood wants to merge 8 commits into
apache:mainfrom
sandugood:feat/non-literal-default-value-window
Open

feat: Enable expressions as default value in lead/lag function#22134
sandugood wants to merge 8 commits into
apache:mainfrom
sandugood:feat/non-literal-default-value-window

Conversation

@sandugood
Copy link
Copy Markdown

@sandugood sandugood commented May 12, 2026

Which issue does this PR close?

Rationale for this change

In the current DataFusion implementation we can only use a ScalarValue as the default value in both LAG and LEAD window functions. That means that users can't reference other columns' values from the table as default value and create various combinations of them.

It means that currently we cannot perform something like:
lead(col1, 1, col1 / col2) as lead_result

What changes are included in this PR?

  1. Added DefaultValue enum that can be either a Literal value or an Expression
  2. Reimplemented parse_default_value function to take default_value's type into consideration
  3. Added two functions: shift_with_array_default and evaluate_all_with_ignore_null_and_array_default for the situations, when default_value is of DefaultValue::Expression type
  4. Added examples for this usecases.

Are these changes tested?

Yes, changes were tested:

  1. Added additional logic tests in window.slt for the sqllogictest

Are there any user-facing changes?

There are user-facing changes in terms of documentation and DataFusion usage (no syntax changes per se). Also, docs were updated

@github-actions github-actions Bot added the functions Changes to functions implementation label May 12, 2026
Comment thread datafusion/functions-window/src/lead_lag.rs Outdated
@sandugood sandugood marked this pull request as draft May 13, 2026 08:18
@github-actions github-actions Bot added documentation Improvements or additions to documentation sqllogictest SQL Logic Tests (.slt) labels May 13, 2026
@sandugood
Copy link
Copy Markdown
Author

Added the tests in the window.slt file

@sandugood sandugood marked this pull request as ready for review May 13, 2026 21:11
SELECT
employee_id,
salary,
lag(salary, 1, 0) OVER (ORDER BY employee_id) AS prev_salary
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still need examples with literal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support ColumnExpr in LAG/LEAD for default parameter

2 participants